Unsupervised Sentence Representation via Contrastive Learning with Mixing Negatives

Unsupervised sentence representation learning is a fundamental problem in natural language processing. Recently, contrastive has made great success on this task. Existing constrastive based models usually apply random sampling to select negative examples for training. Previous work computer vision shown that hard help achieve faster convergency and better optimization learning. However, the importance of negatives yet be explored. In study, we prove are essential maintaining strong gradient signals training process while ineffective representation. Accordingly, present model, MixCSE, extends current state-of-the-art SimCSE by continually constructing via mixing both positive features. The superior performance proposed approach demonstrated empirical studies Semantic Textual Similarity datasets Transfer task datasets.